Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cephfs kafka #4922

Draft
wants to merge 2 commits into
base: edge
Choose a base branch
from
Draft

Cephfs kafka #4922

wants to merge 2 commits into from

Conversation

aduffeck
Copy link
Contributor

@aduffeck aduffeck commented Nov 8, 2024

This PR adds a posixfs watcher for cephfs. It depends on the kafka connector implemented by croit to receive the required fs events.

During a k6 test with a POC cluster it reached 100% success rate and a decent performance despite being a VM-based test instance only.

The configuration used was

      STORAGE_USERS_DRIVER: posix
      STORAGE_USERS_POSIX_ROOT: /var/lib/cephfs/ocis-data/storage-users/
      STORAGE_USERS_POSIX_WATCH_MOUNT_POINT: /var/lib/cephfs
      STORAGE_USERS_POSIX_USE_SPACE_GROUPS: false

      STORAGE_USERS_POSIX_WATCH_TYPE: cephfs
      STORAGE_USERS_POSIX_WATCH_PATH: ocis
      STORAGE_USERS_POSIX_WATCH_FOLDER_KAFKA_BROKERS: 172.31.105.4:909

      STORAGE_USERS_ID_CACHE_STORE: nats-js-kv
      STORAGE_USERS_ID_CACHE_STORE_NODES: localhost:9233

The result of the k6 tests:

     checks.........................: 100.00% ✓ 3644     ✗ 0   
     data_received..................: 1.6 GB  4.1 MB/s
     data_sent......................: 833 MB  2.1 MB/s
     http_req_blocked...............: avg=539.47µs min=1.18µs  med=5.08µs   max=36.02ms p(90)=6.73µs   p(95)=7.68µs  
     http_req_connecting............: avg=205.64µs min=0s      med=0s       max=16.7ms  p(90)=0s       p(95)=0s      
     http_req_duration..............: avg=223.14ms min=21.92ms med=113.1ms  max=13.51s  p(90)=427.67ms p(95)=678.19ms
       { expected_response:true }...: avg=223.14ms min=21.92ms med=113.1ms  max=13.51s  p(90)=427.67ms p(95)=678.19ms
     http_req_failed................: 0.00%   ✓ 0        ✗ 3620
     http_req_receiving.............: avg=10.75ms  min=17.22µs med=79.16µs  max=2.94s   p(90)=129.22µs p(95)=966.74µs
     http_req_sending...............: avg=24.89ms  min=7.94µs  med=29.87µs  max=12.47s  p(90)=40.47µs  p(95)=47.13µs 
     http_req_tls_handshaking.......: avg=327.31µs min=0s      med=0s       max=26.39ms p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=187.49ms min=21.82ms med=108.95ms max=2.56s   p(90)=382.81ms p(95)=611.5ms 
     http_reqs......................: 3620    9.282039/s
     iteration_duration.............: avg=7.17s    min=2.04s   med=2.12s    max=1m16s   p(90)=30.08s   p(95)=30.72s  
     iterations.....................: 2851    7.310246/s
     vus............................: 3       min=0      max=75
     vus_max........................: 75      min=75     max=75

Note: Depends on owncloud/ocis#10519

@aduffeck
Add a watch_mount_point option to posixfs
8aa57bd 
Paths from fs events will be considered relative to this mount point.
@aduffeck
Add a cephfs watcher
4fd321a 
The watched cephfs needs to publish fs to a kafka queue like the croit
implementation does.
@update-docs Update Docs
Copy link

update-docs bot commented Nov 8, 2024

Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@labkode labkode Awaiting requested review from labkode labkode will be requested when the pull request is marked ready for review labkode is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@aduffeck